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INTEGRATED  TEST  AND  ANALYSIS  PROCEDURES  FOR 
SMALL  ARMS  WEAPON  SYSTEMS  EVALUATION 


1 .  INTRODUCTION 


1.1  Purpose 


1 


''^►The  purpose  of  this  document  is  to  summarize  the  findings  of  the 
Small  Arms  Methodology  Study  in  terms  of  a  working  format  for  weapon 
system  evaluation  using  the  three  small  arms  test  facilities.  Section 
4*~a-discussieft-»£?the  major  parameters  of  the  integrated  test  proce¬ 
dure.  Section  2  describes  procedures  for  training  and  scheduling  test 
soldiers  through  the  facilities  taking  into  account  the  two  major  test 
variables,  weapons  and  modes  of  fire.  In  addition,  rationale  is  provided 
for  the  recommended  sample  size  estimation  and  test  soldier  selection 
techniques.  Section  3  describes  the  analytical  procedure  to  be  used  on 
the  field-generated  data.  The  data  are  used  for  two  purposes:  selection 
of  the  most  suitable  combat  weapon  system  and  optimization  of  weapon 
system  performance.  The  procedure  described  emphasizes  the  selection  of 
the  superior  weapon  at  the  earliest  possible  point,  consistent  with  a 
thorough  evaluation  of  operational  performance,  in  order  to  maximize  the 
time  available  for  imoroving  the  selected  weapon's  performance. 

/f 

Analytical  Techniques 
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1.2 


The  analytical  techniques  required  for  the  comparison  of  weapon  per¬ 
formance  and  for  searching  for  areas  in  which  a  potential  for  improved 
performance  exists  are  quite  different.  Hence,  the  earlier  the  weapon 
comparative  analysis  is  completed,  the  more  time  will  be  available  for 
analysis  of  other  factors.  For  example,  data  can  be  massaged  to  determine 
if  sight  alighment  problems  exist,  if  there  is  a  degradation  in  long 
range  hit  probability,  if  the  weapon  has  poor  point  firing  characteristics, 
if  ammunition  consumption  in  a  prolonged  attack  would  approach  the  criti¬ 
cal  point,  and  so  on.  Further,  the  data  base  has  the  potential  to  answer 
operational  questions  such  as:  what  is  the  effective  range  of  the  weapon, 
which  range  should  be  used  to  open  fire,  and  which  mode  should  be  used  in 
the  attack,  defense,  or  quickfire  situation.  This  analytical  approach 
permits  the  maximum  utilization  of  the  available  data  base  which,  in  turn, 
should  result  in  the  selection  of  the  weapon  system  with  greatest  opera¬ 
tional  effectiveness.  Further,  the  operational  characteristics  of  the 
weapon  will  have  been  defined  to  the  extent  possible  within  the  limita¬ 
tions  of  the  available  data. 


1.3  The  Testing  Situation 


1.3.1  Operational  performance  testing  is  designed  to  yield  estimates  of 
combat  effectiveness.  Other  factors  such  as  weapon  signature  and  safety 
are  not  treated  formally  since  these  characteristics  are  evaluated  during 
the  engineering  test  phase.  The  operational  test  employs  three  general 


combat  missions:  attack,  defense,  and  movement  security.  The  attack 
facility  attempts  to  duplicate  such  ccmbat  actions  as  fire  and  movement, 
close  combat,  and  frontal  attack.  Tl.e  Small  Arms  Methodology  Review 
performed  by  USAIB  personnel  found  that  these  three  combat  actions 
contained  the  essential  elements  of  all  combat  actions  associated  with 
attack  situations.  To  perform  these  combat  actions  several  specific 
tasks  must  be  performed  by  the  rifleman.  The  test  facility  is  designed 
to  cause  the  test  soldier  to  perform  these  tasks  in  a  manner  similar  to 
that  required  in  combat.  The  critical  combat  tasks  for  the  attack 
situation  are: 

a.  Long  range  fire  -  unsupported 

b.  Medium  range  fire  increasing  in  volume  -  unsupported 

c.  Intense,  accurate  fire  -  medium  to  close  range 

d.  Rapid  movement 

e.  Rapid  magazine  change 

f.  Rapid  dicplacement 

1.3.2  The  quickfire  facility  attempts  to  duplicate  the  following  combat 
actions:  advance  to  contact,  combat  in  cities,  search  and  clear,  combat 
patrol,  and  reconnaissance  patrol.  The  critical  combat  tasks  that  are 
required  of  the  test  soldier  are: 

a.  Intense  accurate  fire,  medium  to  close  range 

b.  Violent,  quick-reaction  fire 

c.  Alert  movement 

d.  Rapid  reaction 

e.  Immediate  initiation  of  return  fire 

f.  Deliberate,  methodical  movement  w/ detailed  observation 

g.  Anticipated  short  and  medium  range  enemy  contact 

1.3.3  Finally,  the  defense  facility  evaluates  such  combat  actions  as 
retrograde  operations,  deliberate  defense,  hasty  defense,  sniper,  and 
area  or  position  security.  The  critical  combat  tasks  required  of  the 
test  soldier  are: 

a.  Long  range  precision  fire 


b.  Medium  range  precision  fire 

c.  Short  range  precision  fire 

d.  Minimum  exposure 

e.  Sustained  combat  action 

f.  Rapid  magazine  change 

g.  Rapid  reduction  of  malfunctions 

1.3 .4  The  validity  of  the  test  results  produced  in  operational  testing 
is  directly  dependent  on  how  veil  the  test  facilities  succeed  in  dupli¬ 
cating  the  combat  ections  and  tasks.  The  target  presentation  scenarios 
must  be  as  realistic  as  possible;  the  targets  must  behave  os  much  like 
enemy  soldiers  as  possible.  The  scenarios  must  be  sufficiently  long  to 
tax  sustainability  of  the  weapon  system;  this  time  factor  is  an  impor¬ 
tant  consideration  in  the  fire  fight.  To  insure  that  combat  realism 

is  part  of  each  scenario,  optimized  target  presentation  scenarios  have 
been  prepared  for  the  defense  facility  end  planned  for  the  quickfire 
and  attack  facility.  These  should  be  used  whenever  applicable  for  the 
operational  test.  Special  scenarios  may  be  written  and  used  as  required. 
For  instance,  the  testing  of  a  new  sniper  scope  may  dictate  e  special 
target  presentation  scenario  that  emphasizes  presentation  of  long  range 
stationary  and  moving  targets.  The  testing  of  ci-artermaster  equipment, 
such  as  CB  overgarments,  mey  require  emphasis  cn  < arget  acquisition  which, 
in  turn,  may  require  a  special  target  presentation  scenario. 

1.3.5  The  procedure  for  conducting  operational  tests  are  discussed  in 
Section  2..  Techniques  are  described  for  selection  of  the  sample  of  test 
soldiers,  the  assignment  of  test  weapons,  training,  snd  scheduling. 

These  procedures  should  be  followed  as  closely  as  possible  for  each 
operational  service  test. 

1.4  Criteria 

Hie  criteria  for  the  evaluation  of  weapon  system  performance  on 
the  test  facilities  fall  into  four  classes:  accuracy,  sustainability, 
responsiveness,  and,  to  the  extent  possible,  reliability.  Within  these 
classes  several  measures  of  effectiveness  (MOE)  are  used. 

1.4.1  Accuracy  -  The  accuracy  MOE  that  are  available  for  analysis  of 
weapon  performance  are: 

a.  Number  of  hits 

b.  Hit  probability  per  trigger  pull 

c.  First  round  hit  probability 


d.  Engagement  hit  probability 

e.  Distribution  of  near  misses 

1.4.2  Sustainability  -  The  primary  sustainability  KOE  is  the  number 
of  hits  per  pound.  A  secondary  sustainability  KOE  considered  in  this 
analysis  is  hits  per  basic  load.  The  number  of  rounds  in  the  basic 
load  of  the  weapon  system  multiplied  by  the  hit  probability  equals 
total  potential  lethality  in  each  technical  situation. 

1.4.3  Responsiveness  -  The  responsiveness  KOE  are  primarily  associated 
with  the  quickfire  facility  and  ore  particularly  useful  in  pinpointing 
weapon  handling  systems.  These  include: 

a.  Time  to  first  round 

b.  Time  to  first  hit 

c-  Time  between  rounds 

d.  Time  to  shift  fire 

1.4.4  Reliability  -  Three  KOE  are  used  as  a  measure  of  weapon  system 
reliability:  number  of  malfunctions,  number  of  rounds  between  malfunc¬ 
tions,  and  time  to  clc*r  malfunctions.  These  KOE  ere  of  limited  useful¬ 
ness  since  most  weapon.'  that  ’•each  the  operational  testing  phase  of  the 
test  cycle  heve  passed  manj  ngineering  reliability  tests.  Further,  the 
duration  of  the  operatic  test  is  relatively  short  -  each  weapon  sys¬ 
tem  fires  only  a  few  h.oidred  rounds.  This  small  sample  size  increases 
the  difficulty  of  estimating  reliability.  However,  these  data  are 
available  and  should  be  analyzed  to  determine  if  potential  problems 
exist. 

1.5  Analysis 

A  step  by  step  procedure  for  the  comparative  analyses  of  weapon 
system  performance  appears  in  Section  3*  These  steps  are  designed  to 
act  as  a  guide  for  the  analyst,  describing  a  path  through  the  labyrinth 
of  variables.  The  procedure  describes  the  order  of  importance  of  com¬ 
binations  of  variables  and  KOE  with  emphasis  on  specific  decision  nodes. 
The  plan  is  only  a  guide;  deviations  from  the  plan  should  be  made  if  the 
indicators  in  the  data  bsse  pinpoint  areas  of  interest  and  importance 
'  that  were  unforseen  at  the  time  this  plan  was  prepared.  The  analysis 
is  8  multistage  analysis  which  probes  the  data  base  until  one  weapon 
system  proves  superior  to  the  other.  The  point  in  the  analysis  at 
which  superiority  is  shown  is  not  constant  but  depends  on  the  performance 
characteristics  of  weapon  systems.  For  weapons  with  widely  different 
firing  characteristics  superiority  may  be  shown  in  the  initial  phase 
of  the  analysis.  Fbr  closely  competing  weapons,  the  analysis  moves 


to  increasing  depths  in  the  data  basis.  It  is  terminated  when  the 
weapons  are  f^und  to  have  no  operationally  important  performance  differ¬ 
ences.  At  the  point  at  which  superiority  is  found,  regardless  of  the 
level  of  the  depth  of  the  analysis,  emphasis  is  shifted  to  the  task  of 
optimizing  weapon  system  performance.  If  no  differences  are  found,  the 
data  are  used  to  optimize  the  standard  weapon's  performance,  assuming 
that  the  standard  weapon  will  be  selected  as  the  superior  weapon  if  no 
meaningful  differences  o<-cur.  (There  are  no  additional  costs  associated 
with  selection  of  the  standard  weapon. ) 

2.  FIELD  TEST  PROCEDURES 

ftiis  selection  treats  the  six  major  factors  which  are  associated  with 
field  testing:  test  soldier  selection,  sample  size,  weapon  assignment, 
training,  scheduling,  and  implementation  of  the  test. 

2.1  Selection 

Test  soldiers  will  normally  be  supplied  from  Fort  Eenning  units, 
currently  the  197th  Infantry  Brigade.  The  normal  procedure  is  to 
select  either  a  random  sample  or  a  stratified  random  sample.  Using 
stratified  random  sampling,  personnel  records  are  reviewed  end  normally 
the  extreme  cases  are  removed.  For  example,  soldiers  with  exceptional 
high  or  low  IQ's,  aptitudes,  height,  or  weight  are  removed  from  the 
sample  population.  Tne  remainder  are  then  stratified  into  subgroups 
based  on  such  physical  characteristics  as  right  or  left  handedness  or 
visual  acuity.  Soldiers  are  then  selected  by  chance  from  the  subgroups 
to  form  the  groups  of  test  soldiers.  Tne  proportion  of  soldiers  with 
specific  characteristics  in  each  test  group  should  be  similar  to  the 
general  -population  so  that  each  test  groin)  is  a  microcosm  of  the  popu¬ 
lation  of  Infantrymen  as  a  whole.  Using  random  sampling,  individuals, 
fire  teams,  or  squads  are  simply  selected  by  chance  from  the  parent 
unit  in  groups  of  n  soldiers  each.  The  sample  size  is  discussed  in  the 
following  section. 

2.2  Sample  Size 

2.2.1  Hie  knowledge  acquired  thus  far  during  the  methodology  study  has 
contributed  immeasurably  to  the  problem  of  sample  size  estimation. 

Given  a  specific  test  criteria  and  the  selection  of  appropriate  MOE,  the 
re'TJ’ir^d  sample  size  can  be  estimated  reliably.  However,  a  full  scale 
operational  evaluation  of  weapon  performance  will  entail  the  use  of  r-2.1 
test  facilities  and  data  will  be  collected  for  all  small  arms  K02.  Biis 
\equires  a  sample  large  enough  for  the  worst  cases  out  of  several  hundred 
possible  combinations  of  variables.  For  instance,  one  combination  of 
variables  might  be  the  comparison  of  rifle  performance  in  the  semiauto¬ 
matic  mode  on  the  attack  facility  at  $60  meters  using  time  to  first  hit. 
Since  the  number  of  hits  achieved  at  this  long  range  would  be  extremely 
*•13811,  a  large  number  of  trials  and  test  soldiers  would  be  required. 
Consequently,  it  would  not  be  cost  effective  to  plan  for  the  worst  case. 
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Another  raetnod  nusi  be  used. 

2.2.2  Evaluation  of  the  results  of  the  three  major  field  experiments 
of  the  methodology  study  indicate  that  adequate  information  can  be 
collected  with  96  soldier/rifle  systems  of  each  type  being  tested. 

This  number  will  provide  sufficient  replications  for  each  major  cell 
condiction  (i.e.  combination  of  variables)  for  most  measures  of  effec¬ 
tiveness  (MOE).  Inis  suggested  sample  size  is  based  on  an  analysis 
of  the  requirements  for  trie  most  important  1-50E,  number  of  hits. 
(Selection  of  number,  of  hits  as  the  most  important  MOE  is  discussed 
in  Section  3)-  The  rationale  for  the  recommended  sample  of  96  soldier/ 
rifle  systems  of  each  type  is  described  in  paragrepgh  2.2.3. 

2.2.3-  The  sample  size  required  to  conduct  an  experiment  depends  on 
several  considerations.  For  example  the  sample  size  required  to  place 
confidence  intervals  is  determined  differently  than  the  sample  size 
required  to  test  a  given  hypotheses.  The  sample  size  formula  used 
herein  was  selected  because  of  its  revelence  to  a  test  of  hypothesis 
that  -  up.  The  formula  from  Ste^l  and  Torre  is: 

r  >_  2(t0  +  tj)gS2 


Where: 

r  =  The  sample  size  per  test  weapon 

tQ  =  The  t  value  associated  with  the  desired  probability  of 
making  a  Type  I  error 

t^  =  The  t  value  associated  with  the  probability  2(l-P)  where 
P  is  the  required  probability  of  detecting  a  difference  of 
size  ^  if  such  a  difference  exists.  The  t  associated  with 
a  Type  II  error 

S2  =  The  variance  estimate 

<5  =  The  difference  to  be  detected 

In  order  to  supply  tQ  and  t^  values  to  the  formula  e  guess  as  to  what 
sample  size  would  be  required  is  necessary.  A  first  guess  of  60  was 
made.  The  desired  probabilities  of  making  a  Type  I  and  Type  II  error 
must  also  be  established.  The  probability  of  a  Type  I  error  was  chcsen 
as  .05.  This  means  that  when  the  hypothesis  (u^-Ug)  is  trie  it  is 
desired  to  have  only  a  .05  probability  of  rejecting  that  hypothesis. 

The  probability  of  a  Type  II  error  was  chosen  as  .10.  This  means  that 
if  the  difference  between  y  ^  and  u  2  is  as  large  as  5  then  the  proba¬ 
bility  of  accepting  the  hypothesis  (  u^=  )  is  .10.  The  values  for  tQ 

and  were  thus  determined  to  be  2.0  and  I.296  respectively. 
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2.2.4  Observations  vere  taken  froa  Defense  I  Experiment  on  the 
number  of  hits  obtained  by  test  soldiers  during  the  entire  exercise. 

An  estimate  of  the  variance  was  calculated  and  found  to  be  15 8.  Using 
this  and  applying  an  equation  for  sample  size  estimation  recommended 
by  Steel  and  Torre,  the  sample  site  is  estimated  as 

r  =  2(2.0  +  1.289)?  158. 

(6)2 


r  =  95.01  or  96.OO 


when  it  is  desirable  to  be  able  to  detect  a  difference  of  6  hits  between 
two  rifle  systems,  which  represents  approximately  25  percent  superiority 
on  the  defense  facility.  The  difference  observed  between  the  two  rifles 
during  Defense  Experiment  I  was  six  hits  per  14 -minute  trial.  Therefore, 
96  observations  for  each  rifle  should  be  sufficient  for  detecting  differ- 
ences  as  small  as  6  hits  between  competing  veapen  systems.  This  sample 
size  is  considered  the  minimum  size  for  a  full-scale  comparative  test. 
Larger  sample  sizes  will  yield  increasingly  more  usable  information  con¬ 
cerning  vepon  performance  if  the  resources  are  available. 

2.2.5  Figure  1  gives  sample  size  requirements  for  different  values  of  1 
and  three  different  probabilities  of  Type  II  error.  A  second  Iteration 
of  the  process  can  be  performed  using  the  sample  size  of  the  first  itera¬ 
tion  to  look  up  new  values  of  t0  and  t^.  It  was  not  done  here  due  to  the 
snail  changes  In  t  values  when  sample  sizes  of  this  magnitude  are  under 
consideration. 

2.3  Weapon  Assignment 


2.3.1  In  most  tests  where  the  performance  of  people  performing  a  given 
task  with  different  machines  is  the  subject  of  evaluation,  the  best 
procedure  Is  to  rotate  the  people  in  such  a  manner  that  each  person 
uses  each  machine.  This  eliminates  many  experimental  design  problems 
and  insures  that  groups  are  balanced  with  respect  to  the  machines.  In 
small  arms  testing,  people  performance  is  not  the  subject  of  investiga¬ 
tion:  the  primary  interest  is  the  performance  of  the  raan/weapon  system. 
The  rotation  of  test  personnel  from  one  weapon  to  another  may  be  a 
serious  source  of  bias  due  to  unfamiliarity  or  negative  transfer  between 
weapons.  That  is,  if  the  design  of  one  weapon  is  more  familiar  to  the 
test  soldier,  he  may  perform  extremely  well.  When  rotated  to  another 
weapon  of  different  design,  he  may  attempt  to  use  the  weapon  wrongly  or 
he  may  use  it  poorly  simply  because  he  is  less  familiar  with  its 
design.  If  the  performance  of  the  soldier  were  being  evaluated,  rota¬ 
tion  would  insure  an  equal  chance  for  all.  However,  when  performance 
of  the  entire  system  is  of  primary  interest,  rotation  can  introduce  a 
bias. 
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2. 3 *2  The  method  recommended  is  to  divide  the  entire  test  group  into 
two  subgroups  of  equal  capability.  The  method  used  to  divide  the 
groups  can  be  a  random  method,  which  does'  not  insure  equal  groups,  but 
minimizes  the  chances  of  unequal  groups.  Another  procedure  is  to  use 
a  specifically  defined  procedure.  The  recommended  method  is  to  measure 
the  soldier* s  general  shooting  ability  when  he  fires  the  standard  weapon 
on  a  25-meter  range.  Performance  is  measured  by  determining  the  mean 
spread  of  the  shot  group  of  each  soldier  when  he  fires  at  the  Canadian 
bulls-eye.  Select  the  uwo  best  shot  gtoups  and  flip  a  coin  to  assign 
one  test  soldier  to  each  weapon  system.  Follow  the  same  procedure  for 
the  next  two  best  shot  groups,  continuing  until  all  test  soldiers  have 
been  assigned.  Other  means  of  assigning  weapons  may  be  used.  However, 
this  system  has  been  tried  three  times  and  in  each  case  statistical 
tests  were  run  on  performance  data  and  no  significant  differences  were 
found  between  groups  with  respect  to  accuracy  of  fire  on  a  known  distance 
range. 

2.4  Training 

This  variable  was  found  to  be  a  significant  source  of  bias  in 
weapon  system  evaluation.  Accelerated  training  classes  of  up  to  three 
days  were  held  to  insure  that  the  test  soldier  was  familiar  with  the 
new  weapon.  In  analyzing  the  results,  bias  favoring  the  standard 
weapon  was  found.  In  the  time  allotted  it  was  not  possible  to  achieve 
a  level  of  familiarity  between  the  soldier  and  new  weapon  that  was 
comparable  to  the  level  of  familiarity  with  the  standard  weapon.  The 
basic  Infantry  combat  training  and  subsequent  experience  with  the 
weapon  could  not  be  matched  for  the  new  weapon  in  a  3-day  training 
cycle.  Indications  were  that  a  2-week  intensified  course  with  several 
hundred  liye  rounds  would  be  required  before  the  bias  could  be  elimi¬ 
nated.  This  procedure  is  recommended  in  future  operational  evaluations. 

2.5  Scheduling 

2.5.1  After  the  soldiers  have  been  selected,  assigned  to  their  respec¬ 
tive  weapon  systems,  and  trained,  they  must  participate  in  the  test  by 
firing  on  each  of  the  three  test  facilities.  The  optimum  method  of 
accomplishing  the  firing  is  to  allow  one  third  of  the  groups  to  fire  on 
the  defense  facility  first,  then  on  the  attack  facility,  and  finally  on 
the  quickfire  facility.  Another  third  of  the  group  should  begin  with 
the  attack  facility,  and  the  last  third  should-  begin  with  the  quickfire 
facility.  The  entire  schedule  would  be: 

Test  Group 


A 

B 

C 

ATT 

1 

.3 

2 

QF 

2 

l 

3 

DEF 

3 

2 

1 

ar&mrffrff 
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Each  third  (A,  B,  and  C)  fires  in  a  different  sequence.  Each  group 
contains  equal  numbers  of  each  weapon  system.  This  design  balances 
the  effects  of  test  facility  and  allows  comparisons  of  performance 
from  one  facility  to  another  if  desired.  The  method  has  definite 
drawbacks  from  a  practical  standpoint  since  it  would  require  movement 
of  the  ADPE  three  times.  See  Figure  2. 

2.5*2  The  alternate  method  would  be  to  test  all  soldiers  on  one 
facility  and  then  move  to  the  next  which  would  require  only  one  addi¬ 
tional  move.  It  is  possible  that  with  this  procedure  each  test 
soldier  would  improve  in  performance  slightly  as  he  becomes  more 
familiar  with  the  targets  and  cues  of  the  facilities  and  hence 
gathers  increasing  confidence.  If  performance  on  the  last  facility 
should  markedly  improve,  for  instance,  the  improvement  could  not  be 
attributed  to  the  facility's  characteristics  or  the  combat  situation 
because  of  the  confounding  of  the  learning  factor.  Comparisons  between 
facilities  using  such  measures  as  time  to  first  round  would  be  risky 
because  of  the  unbalanced  design. 

2.5.3  Assuming,  then,  that  the  former  method  will  be  employed,  still 
further  balance  is  required.  If  two  or  more  modes  of  fire  are  used, 
these  must  also  be  balanced  in  the  design.  For  example  in  a  two 
weapon-two  mode  test,  group  A  would  consist  of  one  third  of  the  entire 
group  of  the  192  test  soldiers  (using  the  minimum  recommended  sample 
size).  Of  the  64  members,  32  will  be  equipped  with  weapon  X  and  32 
with  weapon  Y.  Each  test  soldier  will  fire  twice  on  each  test  facility, 
once  in  each  of  the  two  modes  (e.g.  serai  automatic  and  controlled  burst). 
'Jiie  schedule  for  the  quickfire  facility  is  shown  in  Figure  3*  The  same 
schedule  would  be  repeated  for  group  members  33 -6b  and  would  be  repeated 
again  changing  the  mode  of  fire  for  each  individual.  This  same  proce¬ 
dure  would  be  used  for  each  of  the  three  groups  of  64  test  soldiers  as 
they  are  rotuted  among  the  three  facilities.  A  total  of  19?  trials 
would  be  run  on  the  quickfire  facility  for  each  weapon  type.  Figure  4 
shows  the-  schedule  for  the  attack  facility.  The  schedule  would  be 
repeated  for  each  group  of  eight  soldiers  in  the  64-man  test  group. 
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TRIAL  GROUP  MEMBER  MODE  TRIAL  GROUP  MEMBER  MODE 


1 

XI 

S 

17 

X5 

S 

2 

Y1 

S 

18 

Y5 

S 

3 

X2 

S 

19 

X6 

S 

4 

Y2 

S 

20 

Y6 

S 

5 

X3 

S 

21 

X7 

S 

6 

Y3 

S 

22 

Y7 

S 

7 

X4 

S 

23 

X8 

S 

8 

Y4 

S 

24 

Y8 

S 

9 

X5 

CB 

25 

XI 

CB 

10 

Y5 

CB 

26 

Yl 

CB 

11 

X6 

CB 

27 

X2 

CB 

12 

Y6 

CB 

28 

Y2 

CB 

13 

X7 

CB 

29 

X3 

CB 

14 

Y7 

CB 

30 

Y3 

CB 

15 

X8 

CB 

31- 

X4 

CB 

16 

Y8 

CB 

32 

Y4 

CB 

Legend : 

X,Y  -  Weapon 

S  -  Semiautomatic 

CB  -  Controlled  Burst 


Quickfire  Schedule 
Figure  3 
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TRIAL* 

GROUP  MEMBERS 

MODE 

1 

XI, 

X2, 

X3, 

X4 

S 

2 

Yl, 

Y2, 

Y3, 

Y4 

S 

3 

X5, 

X6 , 

X7, 

X8 

CB 

4 

Y5, 

Y6, 

Y7 , 

Y8 

CB 

5 

Yl, 

Y2, 

Y3, 

Y4 

CB 

6 

XI, 

X2, 

X3, 

X4 

CB 

7 

Y5, 

Y6, 

Y7, 

Y8 

S 

8 

X5, 

X6, 

X7, 

X8 

S 

♦Trial  -  One  4-member  fire  team  per  trial 

ATTACK  SCHEDULE 
Figure  4 


2.5.4  The  schedule  shown  in  Figure  4  can  also  be  used  for  the  defense 
facility  if  four  firers  are  to  be  used.  Adjustments  in  firing  group 
size  can  be  made  using  up  to  10  firers  at  a  time.  The  same  scheduling 
technique  should  be  employed  when  scheduling  firing  groups .of  different 
sizes. 
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3 .  ANALYTICAL  PROCEDURE 

3.1  General  Description 


3.1.1  This  section  describes  the  recommended  procedure  for  analyzing 
performance  using  data  generated  on  the  Infantry  Board's  three  small 
arms  test  facilities.  The  multi-step  process  is  outlined  and  described 
in  the  following  subsections.  The  flow  chart  in  Figure  5  briefly 
describes  the  decision  nodes  and  the  resulting  analytical  steps.  The 
paragraphs  that  follow  provide  a  general  description  of  the  various 
steps  in  this  procedure. 


3.1.2  The  initial  step,  the  primary  analysis,  is  to  determine  whether 
there  is  a  significant  interaction  between  weapons  and  facilities  using 
the  measure  target  hits.  If  no  interaction  exists  as  determined  by 
an  analysis  of  variance,  the  analyst  then  determines  whether  one  weapon 
system  was  significantly  superior  on  all  test  facilities.  If  one 
weapon  is  superior,  the  analyst  proceeds  directly  to  the  optimization 
analyses,  which  is  described  in  paragraph  3.4.  If  weapons  were  not 
significantly  different,  the  analyst  proceeds  to  the  secondary  analysis, 
which  is  described  in  section  3.3.  The  secondary  analysis  is  designed 
to  identify  small  differences  between  closely  competing  weapons. 


3.1.3  If  an  interaction  was  found  during  the  initial  step,  the  primary 
analysis  must  be  continued  until  the  interaction  is  explained  or  justi¬ 
fied.  An  interaction  occurs  when  the  difference  between  two  weapon 
systems  does  not  remain  consistent  from  one  facility  to  the  next.  The 
detailed- procedures  for  this  analysis  are  described  in  section  3.3. 


3.1.4  Both  the  primary  and  secondary  analyses  are  terminated  when  the 
superior  weapon  has  been  identified.  At  this  point,  the  analysis  shifts 
from  a  comparative  analysis  to  a  performance  optimization  analysis,  as 
shown  in  the  flow  diagram  in  Figure  6  .  The  objective  of  the  performance 
optimization  analysis  is  to  determine,  using  the  data  generated  during 
the  service  test,  the  best  mode  of  fire  for  each  tactical  situation 
(e.g.  assault,  quickfire),  the  optimum  burst  size  for  the  automatic 
mode  engagement,  maximum  effective  range,  and  other  pertinent  operating 
characteristics.  The  data  base  can  be  used  to  pinpoint  strengths  and 
weaknesses  of  the  weapon  system  and,  hence,  will  form  the  basis  for 
recommendations  concerning  employment  of  the  weapon  system  and  future 
improvements.  The  optimization  analytical  procedure  is  described  in 
Section  3.4. 
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3.2  Primary  /‘ns lysis 
3*2.1  Selection  of  MOE 

3.2.1. 1  At  this  initial  stage  of  the  analysis,  the  objective  is  to 
determine  if  an  appreciable  difference  exists  between,  weapon  systems. 

If  such  a  difference  exists,  the  data  base  and  available  time  can  be 
used  to  optimize  weapon  system  performance  rather  than  using  addi¬ 
tional  time  comparing  weapon  performance.  The  earlier  in  the  analysis 
that  the  superior  weapon  system  can  be  identified,  the  more  time  can 
be  made  available  to  improve  weapon  system  performance. 

3.2. 1.2  The  primary  analysis  employs  a  single  measure  of  effective¬ 
ness.  The  most  valid  single  combat  measure  is  the  ratio  of  friendly 
to  enemy  kills.  Since  the  number  of  friendly  kills  is  a  function  of 
the  effectiveness  of  enemy  weapons  and  the  ability  of  the  enemy  to 
locate  friendly  soldiers,  this  factor  can  be  considered  a  constant 
providing  one  candidate  weapon  does  not  require  more  exposure  on  the 
part  of  the  user  or  have  a  more  identifiable  signature  than  other 
candidate  weapons.  The  manner  in  which  hand  held  rifles  are  used 

is  such  that  no  appreciable  differences  in  exposure  have  occurred  to 
date.  Further,  weepon  signature  is  carefully  evaluated  during  engineer¬ 
ing  tests  to  insure  that  excessive  blast,  smoke,  or  flash  are  minimal. 

3.2.1. 3  Consequently,  one  primary  measure  of  combat  effectiveness  is 
the  number  of  enemy  casualties.  This  type  of  measure  is  known  as  a 
measure  of  mission  accomplishment  where  the  mission  is  to  close  with 
and  defeat  the  enemy.  Tne  small  arms  test  facilities  are  designed 
•and  constructed  to  represent  a  slice  of  life;  they  simulate,  to  the 
'extent  possible,  real  combat  situations.  The  facilities  require 

the  test  soldier  to  perform  the  same  combat  tasks  (e.g.  long  range  aimed 
fire,  quick  reaction  short  range  fire)  required  in  combat.  Because 
of  these  factors,  the  mission  accomplishment  measure  can  be  equated 
to  the  number  of  targets  hit.  As  long  as  target  presentation  scenarios 
provide  realistic  presentations  and  cues,  the  validity  of  the  relation¬ 
ship  between  enemy  casualties  and  target  "casualties"  will  remain  high. 

3. 2. 1.1*  Tfaere  are  many  advantages  to  using  a  mission  accomplishment 
measure.  Almost  all  MOE  are  implicitly  included  in  the  mission 
accomplishment  measure  with  respective  weighting  similar  to  their 
relationship  to  real  combat  measures.  For  instance,  targets  presented 
in  a  realistic  manner  at  all  range:-  consistent  with  the  tactical 
situation  will  insure  that  accuracy  measures  such  as  hit  probability 
are  appropriately  accounted  for  in  the  mission  accomplishment  measure. 
Realistic  scenario  length  will  insure  that  sustainability  is  appropriately 
weighted  on  the  mission  accomplishment  measure.  Weapons  that  are 
unresponsive  and  are  difficult  to  handle  due  to  such  problems  os  poor 
pointing  characteristics  or  excessive  recoil  will  achieve  fever  hits 
on  realistic  targets  and  hence  will  impact  on  mission  accomplishment. 


Since  some  specific  operating  characteristics  cn h  as  optimum  burst 
size  ore  not  yet  known,  the  new  weapon  will  be  at  3  s?,  :;%t  disadvantage. 
Most  probably,  the  new  weapon  will  be  fired  in  a  manner  similar  to  that 
used  with  the  standard  weapon,  which  may  not  be  optimum  for  the  new 
weapon.  A  burst  analysis  is  recommended  regardless  cf  the  test  outcom'" 
to  determine  wheuher  a  significant  bias  exists.  If  une  burst  analyrt" 
shows  a  significant  bias,  that  portion  of  the  service  nest  affected 
(i.e.  the  situation  which  uses  automatic  fire)  must  be  repeated. 

3.2.1. 5  ftiere  are  other  advantages  to  using  a  mission  acco-^.Aishment 
measure.  Weapons  of  grossly  different  characteristics  car  be  compared 
directly.  For  instance,  a  high  rate  of  fire,  multiple  projectile, 
controlled  burst  rifle  could  be  compared  directly  to  ?  conventional 
bolt  action  rifle  using  the  most  valuable  single  index  of  performance: 
the  amount  of  damage  inflicted  on  the  enemy'.  Also.,  the  data  are  more 
suitable  for  use  by  other  agencies  in  such  studiu-s  as  vulnerability  or 
two  sided  combat  simulation  analyses. 

3. 2.1. 6  Mission  accomplishment  (MA)  is  defined  to  be  the  number  of 
targets  hit  in  a  given  tactical  scenario.  This  measure  is  an  encom¬ 
passing  measure  and  has  buried  within  it  the  effects  of  most  of  the 
measures  now  considered  important.  Table  summarizes  the  measures  of 
effectiveness  (MOE)  and  indicates  either  their  inclusion  or  exclusion 
from  MA.  The  rationale  for  the  relationship  between  MA  and  MOE  is  given 
by  MOE  in  subsequent  paragraphs. 

(1)  Distribution  of  near  misses.  This  MOE  is  not  related  to 
MA  as  defined  above. 

(2)  Hit  probabilities  per  trigger  pull  --  semiautomatic  mode 

(3)  Hit  probabilities  per  trigger  pull  —  automatic  mode 

MA%J  [(Trigger  Pulls  at  Range  i)  x  (Ph  at  Range  i)  - 
Duplicate*  Target  Kits] 

(*0  Engagement  hit  probabilities. 

MAJfcE  [(Engagement  at  Range  i)  x  (Engagement  Hp  at  Range  i)  x 
(Average  number  of  targets  hit  per  engagement;] 

♦The  "approximately  equal  to"  is  caused  by  the  existence  of  time 
constraints  on  target  exposure. 

(5)  Time  to  first  hit  (Targets  react  to  hits).  If  two  weapon 
systems  under  test  have  different  time  to  first  hits,  i.e., 
wa  >  v b  w-'iere  Ma  an<i  Ub  ure  the  average  time  to  first  hits  per 
system,  then  MA  i"  affected  because  of  the  limited  target  expo¬ 
sure  time.  Consider  a  distribution  of  time  to  first  hit  for 
weapon  A  as  depicted  in  Figure  7  and  a  distribution  of  time  to 
first  hit  as  depicted  in  Figure  8. 
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Table  I 


Quick 

Attack  Fire  Defense 


MOE 

Distribution  of  near  misses  00  0 


Hit  probabilities  per  trigger 
pull — semiautomatic 

Hit  probabilities  per  trigger 
pull — automatic  mode 

Engagement  hit  probabilities 

Time  to  first  hit 

Time  to  first  round 

Time  between  rounds 

Time  between  bursts 

Time  between  hits 

Time  to  shift  fire 

Number  of  rounds  to  first  hit 

Hits  per  pound 

Time  to  change  magazine 

Time  to  recharge  magazines 

Time  to  clear  malfunctions 

Time  between  malfunctions 

Movement  times 

Compatibility  with  other  equipmart 
Ease  of  handling  in  the  assault  role 
Sound  level  recording  (bli'.st) 

Light  reduction  (smoke  o.i-i  nar.e) 
Visual  light  emission  (flash) 
Ejection  patterns 


XX  X 

XX  X 

XX  X 

XX  X 

XX  X 

XX  X 

XX  X 

XX  X 

XX  X 

XX  X 

X  0  X 

X  X  X 

0  0  0 

XX  X 

X  0  X 

X  0  0 

XX  0 

X  0  0 

0  0  0 

X  0  X 

0  0  X 

X  0  X 


19 


Assusing  roughly  the  sane  standard  deviation  for  both  distribu¬ 
tion  it  is  evident  from  Figures  7  and  8  that  the  curve  in  Figure 
7  has  more  area  to  the  right  of  the  target  exposure  time  and  as 
a  consequence  fever  hits  would  be  achieved  resulting  in  a  lover 

MA. 

(6)  Time  to  first  round.  This  I-JOE  is  related  to  MA  in  the 
sense  that  time  to  first  round  is  correlated  with  time  to  first 
hit.  The  correlation  was  found  to  be  approximately  .65  in  the 
quickfire  experiment  1. 

(7)  Time  between  rounds.  IMs  MOE  is  related  to  MA  to  the 
extent  that  a  long  time  between  rounds  for  one  veapon  system 
would  indicate  interface  problems  and  0  failure  to  get  rounds  down 
range  decreases  the  value  of  MA.  Given  that  each  round  has  a 
potential  contribution  to  MA,  any  increase  in  T3R  will  reduce  MA. 
'Hie  quickfire  experiment  showed  a  minus  correlation  between  total 
rounds  fired  and  time  between  rounds;  totsl  rounds  fired  had  a 
high  positive  correlation  with  effectiveness  as  defined  by  the 
discriminant  function. 


(8)  Time  between  burst.  Same  as  7t 


(y)  Time  between  hits.  Since  targets  are  normally  exposed 
for  a  selected  period,  an  increase  in  time  between  hits  will 
decrease  mission  effectiveness.  The  logic  follows  that  described 
for  MOE  number  7,  above. 

(10)  Time  to  shift  fire.  This  K03  is  a  good  indicator  of  man/ 
weapon  interface  problems.  A  weapon  system  that  performs  poorly 
with  respect  to  this  variable  would  decrease  MA,  because  a 
soldier  is  not  placing  fire  on  new  targets  as  soon  as  he  could. 

(11)  Humber  of  rounds  to  first  hit.  Related  to  item  number 
5,  above. 

(12)  Kits  per  pound.  This  is  a  meesure  of  total  potential 
weapon  system  lethality  expressed  as  a  function  of  total  system 
weight.  It  is  part  of  the  mission  accomplishment  measure  to 
the  extent  that  the  test  scenarios  are  of  realistic  length.  If 
scenarios  are  patterned  after  realistic  defensive  and  attack 
actions,  this  measure  will  affect  the  mission  accomplishment 
measure.  As  ammunition  begins  to  run  low,  the  test  soldier 
will  reduce  his  firing  r3te  to  conserve  ammunition.  As  the  rate 
is  reduced  or  if  the  test  soldier  expects  all  of  his  basic  load, 
the  nunber  of  targets  hit  will  be  reduced  thus  an  impacting  is 
felt  on  the  mission  accomplishment  meesure. 

(13)  Time  to  change  magazines.  If  magazine  change  time  takes 
longer  for  one  weapon  and  the  scenario  is  sufficiently  taxing 
then  this  shortcoming  would  be  reflected  in  MA  for  that  weapon. 

(14)  Time  to  recharge  magazines.  Hot  related  to  MA  due  to  the 
manner  by  which  this  MOE  is  collected. 

(15)  Time  to  clear  malfunctions.  An  undesirable  weapon  per¬ 
formance  on  anyone  of  MOE  15  or  16  has  the  effect  of  taking 
the  weapon  of  action  thus  causing  a  reduction  in  MA. 

(16)  Humber  of  rounds  between  malfunctions.  See  item  1>- 

(17)  •  Movement  times.  Any  weapon  characteristic  that  effects 
movement  time,  ease  of  handling  in  the  assault  and  perhaps 
compatibility  with  other  equipment  would  cause  a  degradation 
in  MA  on  the  attack  facility,  A  slight  problem  reflected  by 
these  MOE  would  probably  be  non-detectable  by  MA. 

(18)  Compatibility  with  other  equipment.  See  item  17. 

(19)  Ease  of  handling  in  the  assault  role. 
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(20)  Sound  level  recording.  Not  part  of  MA. 

(21)  Light  reduction  (smoke  and  haze).  A  weapon  performance 
that  in  any  manner  obscures  targets  would  cause  a  reduction  in 
MA  for  that  weapon. 

(22)  Visual  light  emission  (flash).  This  could  effect  the  MA 
for  night  test  by  affecting  the  individual's  darkness  adqption. 

(23)  Ejection  Patterns.  This  MOE  would  only  effect  MA  if  the 
pattern  caused  the  firer  or  adjacent  firer  any  discomfort  that 
would  tend  to  reduce  his  effectiveness. 

3. 2. 1.7  Since  mbst  of  the  MOE  are  on  integral  part  of  the  primary  measure, 
a  weapon  system  that  proves  to  be  superior  in  terms  of  the  mission  accom¬ 
plishment  measure  on  all  test  facilities  may  be  selected  immediately. 

There  is  no  need  to  continue  the  comparative  analysis  since  the  superior 
weapon  system  has  been  identified.  Qnphasis  should  be  placed  on  the 
analysis  of  various  MOE  to  determine  whether  performance  could  be  im¬ 
proved  even  more.  For  instance,  although  hit  probabilities  will  be  ade¬ 
quate  at  all  ranges,  they  should  be  examined  to  determine  whether 
improvements  are  possible.  Time  between  bursts  should  be  examined  to 

see  if  modifications  could  improve  the  soldier's  ability  to  cope  with 
recoil,  reacquire  the  target,  or  lay  the  sights  more  quickly  and  effec¬ 
tively.  Other  areas  where  potential  increases  are  possible  are  in 
>  optimizing  burst  size,  mode  of  fire,  and  training  procedures.  This 
type  of  analysis  which  is  designed  to  optimize  weapon  system  performance 
differs  from  the  comparative  analysis  of  weapon  system  performance. 
Therefore,  it  is  important  to  determine  as  quickly  as  possible  which 
candidate  weapon  is  superior  in  order  to  maintain  proper  orientation 
of  the  analysis:  optimization  vs  selection. 

3. 2. 1.8  Table  1  shows  that  3  measures  are  not  accounted  for  in  the  MA 

measure:  round  level,  time  to  recharge  magazines,  and  near  miss  dis¬ 

tance.  Engineering  and  safety  tests  normally  monitor  such  weapon  charac¬ 
teristics  as  signature  effects  to  insure  that  the  weapon  is  safe  to  fire. 
The  tactical  significance,  however,  is  not  accounted  for  objectively. 
Military  experience  must  be  used  to  evaluate  the  effects  of  differing 
signature  characteristics  as  these  characteristics  affect  the  vulnera¬ 
bility  of  the  individual,  the  probability  that  he  will  be  detected  by 
the  enemy.  Time  to  recharge  magazines  can  easily  be  measured  as  maga¬ 
zines  are  being  prepared  for  use  in  the  tactical  test.  If  significant 
differences  are  found,  these  differences  should  be  noted  by  the  test 

.  officer.  The  impact  on  the  tactical  situation  can  be  estimated  by  re¬ 
quiring  test  soldiers  to  load  their  own  magazines  during  the  simulated 
fire  fight  on  the  defense  facility.  Finally,  miss  distance  is  not 
included  in  the  MA  measure  since  suppression  is  not  a  parameter  of  the 
test  facilities:  however,  these  data  can  be  used  in  the  secondary  analy¬ 
sis  as  a  part  of  the  supplementary  accuracy  analysis. 
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3.2.2  Technical  Approach 

The  technique  for  performing  the  primary  analysis  is  a  3*2x2  fac¬ 
torial  experiment.  The  factors  are  facilities,  weapons  (assuming  2 
weapons)  and  modes  of  fire  (assuming  2  modes).  The  linear  model  is: 

*UM.  '  u  +  %+  WJ  +  Mk  +  (™>ij  +  (™>ik  +  0*>Jk  + 


''ijk  +  eijkl 


Where: 


u  =  the  overall  mean 


Fi  =  i  facility  effect 
Wj  =  weapon  effect 


=  k  mode  effect 


Yi1kl  ~  observation  corresponding  to  the  1th  soldier  using  the 
kth  mode  of  the  j  h  weapon  on  the  ith  facility 

+v* 

ei1kl  =  random  error  corresponding  to  the  1  soldier  using  the 
k^b  mode  of  the  j™5  weapon  on  the  i**  facility 


The  analysis  of  variance  to  be  performed  is  shown  in  Figure  9.  The 
paragraphs  below  are  referenced  in  Figure  6  and  describe  the  meaning 
of  a  significant  difference  and  the  steps  to  be  followed  should  the  F- 
value  for  a  variable  or  combination  of  variables  be  significant. 
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Primary  Analysis  of  Variance 
Using  Hits  on  Targets 

Figure  9 


♦Significance: 

A  -  A  significant  F-value  for  facilities  simply  means  that  more  hits 
are  obtained  on  one  facility  when  compared  to  the  other  facili¬ 
ties.  This  difference  is  due  to  the  difference  in  number  of  tar¬ 
gets  exposed  and  time  of  exposure  and  has  no  meaning  in  terms  of 
weapon  evaluation. 

B  -  A  significant  F-value  for  weapons  states  that  one  weapon  system 
has  overall  superiority  to  the  other  weaponfs^  on  the  facilities. 
This  finding  is  based  on  the  assumption  that  the  facilities  rep¬ 
resent  a  realistic  slice  of  life  and  that  other  factors  such  as 
weapon  signature  and  exposure  have  been  considered.  Ihis  test 
also  assumes  that  weapons  submitted  to  operational  effectiveness 
testing  have  achieved  an  acceptable  level  of  reliability  and 
durability. 
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C  -  A  significant  value  for  mode  states  one  mode  is  overall 
superior  to  another  on  the  test  facilities  and  initially 
indicates  that  the  weapons  should  be  used  in  the  superior 
mode  at  all  times. 

D  -  A  significant  F-value  for  this  interaction  implies  that  one 
rifle  may  not  be  consistently  superior  on  all  test  facili¬ 
ties,  i.  e.,  one  weapon  may  be  superior  under  one  combat 
condition  but  inferior  in  another.  This  result  will  require 
continuation  of  the  Primary  Analysis  and  is  discussed  in  the  • 
next  subsection,  3.2.3. 

E  -  A  significant  F-Value  indicates  that  one  mode  is  possibly 
superior  on  one  facility  but  inferior  on  another.  Proceed 
to  the  secondary  analysis  since  the  results  indicate  the 
optimum  mode  for  each  facility  or  combat  situation. 

F  -  A  significant  F-value  indicates  that  one  weapon  is  possibly 
superior  in  one  mode  but  inferior  in  the  other  mode.  If  B 
above  was  significant,  the  next  step  is  to  proceed  to  the 
optimization  analysis.  If  B  was  not  significant,  the  next 
step  is  to  begin  the  secondary  analysis. 

G  -  A  significant  F-value  here  is  more  difficult  to  interpret. 

It  indicates  a  significant  interaction  between  the  three 
variables  and  is  known  as  a  second  order  interaction.  If 
this  should  occur  a  specific  analysis  will  have  to  be  under¬ 
taken  to  determine  the  cause.  At  times,  the  second  order 
interaction  may  be  explained  in  terms  of  the  first  order 
interactions.  The  exact  analytic  approach  will  depend  on 
the  first  order  interactions  and  the  composition  of  the  data 
base.  .  Decisions  concerning  the  method  of  reduction  of  the 
interaction  effect  should  be  made  after  the  interaction  occurs. 
It  is  not  practical  to  try  to  account  for  all  possible  causes 
of  possible  second  order  interactions. 

3.2.3  Interpretation  of  the  Interaction 

3. 2. 3.1  This  stage  of  the  analysis  was  reached  because  of  a  signifi¬ 
cant  interaction  between  weapons  and  facilities  as  indicated  by  a 
significant  F-value  in  the  initial  analysis  of  variance  test  (the  F- 
value  of  the  D  or  G  analysis  Figure  9).  If  no  interaction  occurred  at 
D  or  G,  the  analyst  should  proceed  directly  to  the  secondary  analysis, 
section  3.3.  The  interaction  will  have  occurred  if  one  weapon  proves 
superior  on  one  facility  but  inferior  on  one  of  the  other  facilities. 

3. 2. 3. 2  The  first  step  in  this  phase  of  the  analysis  is  to  graph  per¬ 
formance  means  using  number  of  hits  for  the  three  test  facilities,  if 
an  interaction  was  present,  the  resulting  graph  l'  ould  be  similar  to 


one  or  more  of  the  types  of  interactions  shown  in  FigureslO.il  and  12- 

3. 2. 3. 3  FigurelO  is  called  a  Type  A  interaction  and  shows  that  one 
weapon  system  is  consistently  superior  to  the  other  weapon  system.  If 
this  type  of  interaction  occurs  the  analyst  may  proceed  directly  to 
the  Optimization  Analysis. 

3. 2. 3. A  Figure]!  shows  two  possible  Type  B  interactions.  This  type  of 
interaction  indicates  that  there  was  a  significant  difference  in  per¬ 
formance  between  the  competing  weapon  systems,  but  the  differences  were 
not  consistent  across  facilities.  The  analyst  at  this  point  must 
determine  whether  an  explanation  exists  for  these  results.  For  instance, 
referring  to  the  left  hand  graph  in  Figure n,  perhaps  rifle  A  had 
extremely  good  point  fire,  close  range  effectiveness  which  could  be 
expected  from  a  shotgun  or  other  multiple  projectile  weapon.  The 
facility  (F2)  would  be  the  quickfire  facility.  The  decisions  must  now 
be  made  whether  the  improvement  in  the  quickfire  role  is  operationally 
significant,  that  is,  large  enough  to  compensate  for  the  poor  perfor¬ 
mance  in  the  attack  and  defense  roles.  Such  a  decision  would  be  based 
on  the  frequency  of  the  occurrence  of  the  types  of  combat  actions.  Such 
operations  as  fire  and  movement,  hasty  defense,  retrograde  operations, 
and  deliberate  defense  occur  far  more  frequently  than  such  quickfire 
operations  as  combat  in  cities.  If  it  is  felt  at  this  time  that  the 
differences  are  operationally  significant,  the  analyst  must  proceed  to 
the  secondary  analysis.  If  the  differences  are  not  operationally 
significant,  the  superior  weapon  should  be  selected,  and  the  analyst 
should  proceed  to  the  optimization  analysis. 

3. 2. 3. 5  Figure  12  shows  the  Type  C  interactions.  These  interactions  are 
similar  to  the  Type  B  interactions,  but  the  differences  in  performance 
are  of  much  smaller  magnitude.  No  significant  differences  were  found 
between  weapons  although  there  was  a  significant  interaction.  Essentially 
the  same  decision  must  be  made  here  as  was  described  in  the  preceding 
paragraph.  However,  in  this  case,  there  should  be  a  strong  tendency  to 
reject  the  new  weapon  system  in  favor  of  the  standard  weapon.  If  this 

is  done,  the  secondary  analysis  may  be  eliminated  and  the  optimization 
analysis  selected. 

3. 2. 3. 6  If  the  graphing  technique  fails  to  explain  to  the  source  of 
the  interaction,  a  second  procedure  is  recommended.  The  procedure  is  to 
construct  a  two-way  table  (Figure  13)  with  entries  equal  to  the  sum  of 
the  observations  for  each  weapon/ facility  (WF^j)  combination  where  i 
stands  for  the  ic^  weapon  and  j  for  the  jc^  facility. 


26 


The  next  step  is  to  compute  the  sums  of  squares  for  each  sample  effect. 
This  is  accomplished  by  using  the  following  functional  relationships: 

a.  Weapons  with  Attack  Facility 

ss  *  Vi "  HiV2 

2  r  m 

b.  Weapons  with  Quickfire  Facility 

2 

SS  =  (w  F  -  W  F  ) 

22  12 

2  r  in 

c.  Weapons  with  Defense  Facility 
ss  =  (\}v  -  W^Fj2 

2  r  ra 

where  r  is  equal  to  the  number  of  test  soldiers  in  the  test  condi¬ 
tion  and  m  equals  the  number  of  firing  modes  used.  Each  of  the  ss 
values  is  tested  for  significance  with  the  error  value  which  appears 
in  Figure  6  using  an  F-test.  If  no  significant  differences  are  detected, 
the  performance  of  the  competing  weapons  systems  is  essentially  identical 
on  all  three  facilities  with  respect  to  target  hits.  There  is  no  justi¬ 
fication  for  selecting  the  test  weapon  in  terms  of  performance.  Conse¬ 
quently,  based  on  the  operational  testing,  the  standard  weapons  should 
be  retained.  The  data  base  should  be  analyzed  for  possible  inconsis¬ 
tencies  in  the  standard  weapons's  performance  using  the  optimization 
analysis. ' 

3. 2. 3. 7  If  there  are  significant  differences,  the  particular  test  will 
indicate  where  the  inconsistency  is,  i.e.,  on  which  facility(s)  the 
comparative  performance  differed.  Such  inconsistencies  will  necessitate 
moving  to  the  next  phase  of  the  analysis,  the  secondary  analysis. 
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3.3  Secondary  Analysis 


3.3.1  Selection  of  MOE 

3. 3. 1.1  The  second  step  in  the  analysis  occurs  when  the  performance 
of  competing  weapon  systems  is  found  to  be  very  close.  For  example, 
this  will  occur  if  one  weapon  system  is  found  to  be  superior  in  the 
defensive  situation  and  inferior  in  the  attack  situation  with  no 
logical  explanation  for  the  interaction  effect.  Several  selected  MOE 
are  analyzed  in  a  series  of  ana tytical ' steps  (see  Figure  5). 

3. 3. 1.2  The  initial  step  is  the  sustainability  analysis  which  pro¬ 
vides  an  estimate  of  the  staying  power  of  the  weapon  system  in  addition 
to  that  which  is  built  into  the  primary  MOE  via  the  target  presenta¬ 
tion  scenario.  Several  MOE  could  be  selected  including  number  of 
trigger  pulls,  hits  per  pound,  hits  per  basic  load,  number  of  trigger 
pulls  per  basic  load,  and  the  number  of  combat  minutes  per  basic  load. 
From  these,  the  measure  hits  per  pound  has  been  selected  as  the  most 
meaningful  measure  since  it  incorporates  two  weapon  characteristics, 
weight  and  accuracy.  A  soldier's  combat  ef fectiveness  consists 
primarily  of  what  he  can  carry  into  combat;  the  lethality  per  pound 

of  resource  becomes  the  most  meaningful  measure  and  permits  direct 
comparison  of  varied  ammunition  types  in  terms  that  are  meaningful  in 
the  combat  environment.  A  second  measure  has  been  added  as  a  final 
check,  the  number  of  potential  hits  per  basic  load.  Ordinarily  the 
hits  per  pound  measure  will  provide  the  information  required.  However, 
if  the  weapon  component  of  the  man/weapon  systems  being  compared  differ 
markedly  in  weight,  the  difference  will  normally  be  accounted  for  in 
terms  of  the  weight  of  ammunition  carried.  Therefore,  it  is  necessary 
to  know  the  number  of  pounds  carried  in  the  basic  load.  A  more  descrip¬ 
tive  method  of  treating  this  variable  is  to  express  it  in  terms  of 
potential  hits  per  basic  load.  If  the  lethality  of  the  ammunition 
varies  between  weapons,  the  kill-given-a-hit  probability  will  have  to 
be  added  to  make  the  measure  even  more  meaningful.  The  measure  then 
becomes  kills  per  basic  load. 

3.3. 1.3  The  next  step  is  an  accuracy  analysis.  This  has  been  accom¬ 
plished  to  a  large  extent  during  the  initial  analysis  using  the  primary 
MOE.  However,  in  the  case  of  close  competing  weapons,  it  is  necessary 
to  insure  that  effectiveness  is  consistent  over  a  broad  spectrum  of 
range  distances.  If  the  scenarios  are  poorly  designed,  it  would  be 
possible  to  get  an  overkill  on  targets  at  close  ranges  which  would 
inflate  the  number  of  hits  per  trial  or  per  pound  in  the  sustainability 
analysis.  An  effective  all-purpose  weapon  mus-:  provide  effectiveness 
at  all  ranges  associated  with  Infantry  combat.  The- measure  used  in 

this  supplementary  accuracy  analysis  is  hit  probability  per  trigger  pull, 
which  permits  a  direct  comparison  of  weapon  performances  which  is 
independent  of  the  number  of  projectiles  per  round,  the  number  of  rounds 


per  burst,  and  so  on.  A  given  weapon  system  with  its  basic  load  has 
n  number  of  trigger  pulls.  The  number  of  trigger  pulls  multiplied  by 
the  hit  probability  equals  the  expected  number  of  kills  for  a  given 
range.  Comparative  performance  at  all  ranges  will  be  the  basis  of 
the  accuracy  analysis. 

3. 3. 1.4  A  second  measure  of  accuracy  effectiveness  is  near  miss  dis¬ 
tance.  If  performance  is  essentially  equal  in  terms  of  target  kills, 
the  next  most  important  item  is  fire  suppression  which  can  be  defined 
as  a  temporary  combat  kill.  However  the  parameters  associated  with 
suppression  are  not  well  understood.  Thus  far  the  analysis  has  based 
selection  of  weapons  on  the  criteria,  number  of  targets  hit,  because 
of  the  assumption  that  the  realistic  test  facilities  provide  a  posi¬ 
tive  relationship  between  target  hits  and  the  number  of  enemy  soldiers 
that  would  be  hit  in  similar  combat  situations.  There  is  no  known 
method  for  equating  near  misses  or  fire  suppression  to  the  survival 
probability  of  enemy  or  friendly  troops.  Therefore,  suppression  (or 
near  misses)  is  not  recommended  as  a  criteria  for  weapon  selection 

at  this  time.  The  near  miss  data  are  recommended  for  use  during  the 
optimization  analysis  as  an  indicator  of  weapon  accuracy.  This 
measure  of  dispersion  should  be  useful  in  isolating  interface  problems 
such  as  difficulty  in  achieving  a  fine  sight  picture. 

3. 3. 1.5  Should  comparative  performance  remain  very  close  between 
competing  weapons,  it  is  necessary  to  proceed  to  the  next  analytic 
step,  the  responsiveness  analysis.  Responsiveness  is  primarily 
measured  or.  the  quickfire  facility  and  is  used  to  determine  handling 
characteristics  of  the  weapon.  As  mentioned  in  Section  1,  several 
measures  are  available.  The  measures  selected  are  time  to  first  round, 
time  to  first  hit,  time  to  shift  fire,  and  time  between  trigger  pulls. 
Each  provides  an  estimate  of  man  weapon  performance  in  terms  of  the 
man/weapon  interface.  The  first  measures  ease  of  alignment  and  accuracy 
the  second  measures  ease  of  target  reacquisition  and  realignment;  the 
third  measures  how  well  the  soldier  copes  with  recoil  and  target  obscura 
tion,  especially  in  the  automatic  mode. 

3.3. 1.6  At  this  stage  of  the  analysis,  if  no  performance  differences 
are  found  on  which  justification  for  weapon  selection  can  be  made, 
the  weapons  can  be  assumed  to  be  equally  effective  in  combat.  If  no 
other  rationale  exists  such  as  reliability,  cost,  or  safety  for  weapon 
selection,  the  standard  weapon  becomes  the  "superior"  weapon  simply 
because  it  is  already  available  and  in  the  inventory  without  the 
tremendous  costs  associated  with  the  procurement  of  a  new  weapon  system. 
Although  not  shown  on  the  flow  chart,  reliability  is  measured  by  thc- 
nuirber  of  rounds  between  malfunctions.  These  data  do  not  enter  into 
the  decision  criteria  because  of  lack  of  sample  size.  Reliability, 
number  of  rounds  between  failures  (NRBF)  will  be  measured  far  more 
accurately  during  the  engineering  test  phase  and  other  phases  of  the 
service  test.  Consequently,  it  is  assumed  that  a  reliable  weapon  would 
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not  have  reached  the  operational  test  phase.  However,  a  reliability 
analysis  should  be  done  with  NRBF  to  insure  that  an  operational  re¬ 
liability  problem  has  not  been  overlooked.  Any  reliability  problem 
that  occurs  with  this  relatively  small  sample  of  live  fire  should  re¬ 
ceive  major  consideration  and  should  be  resolved  before  any  decision 
is  made  to  select  a  superior  rifle  system  based  on  operational  per¬ 
formance  . 

3.3.2  Technical  Approach 

3. 3. 2.1  Sustainability  Analysis 

Since  one  test  weapon  failed  to  establish  a  clear  superiority 
over  the  competing  weaponfs) ,  the  next  effort  is  to  move  to  more 
detailed  analyses  using  other  MOE .  With  no  difference  in  hits,  the 
number  of  potential  hits  that  a  soldier  can  carry  into  combat  be¬ 
comes  an  important  criterion.  In  the  case  of  sustained  fire  fights 
or  in  situations  where  resupply  becomes  difficult  or  costly,  sustain¬ 
ability  assumes  increasing  importance.  Therefore,  although  there 
were  no  differences  using  target  hits  on  the  three  test  facilities 
even  though  the  trials  were  sufficiently  long  to  duplicate  combat 
fire  fights,  there  is  some  desirability  and  utility  value  in  having 
a  good  ammunition  supply. 

Figure  14  is  a  summary  flow  chart  of  the  sustainability  analysis 
The  initial  test  is  to  compute  the  number  of  hits  per  pound  of  ammuni 
tion.  The  analysis  of  variance  statistical  test  that  is  shown  in 
Figure  ?  is  usee.  The  entire  2x2x3  factorial  analysis  is  repeated. 

If  the  analysis  still  fails  to  show  a  clear  superiority,  a  comparison 
of  rifle  hits  per  basic  load  is  made  again  repeating  the  entire  sta¬ 
tistical  procedure.  Hits  per  basic  load  yields  the  number  of  po¬ 
tential  hits  in  the  basic  load  and  weights  the  outcome  in  fa«or  of 
the  largest  basic  load.  All  other  factors  such  as  lethality  are 
assumed  to  be  equal.  If  lethality  is  not  equal  but  known  from  labora 
tory  tests,  the  number  of  potential  kills  per  basic  load  can  be  esti¬ 
mated  and  used  instead  of  hits  per  basic  load.  The  equation  below 
can  be  used  to  determine  the  kills  per  basic  load  for  each  man.  The 
analysis  proceeds  as  in  the  primary  analysis  previously  outlined. 

Kills  *  I*h  x  lethality  x  basic  load 

where  lethality  =  Number  of  kills 
Number  of  hits 

If  at  the  conclusion  of  the  sustainability  analysis,  no  clear  cut 
difference  between  weapons  has  been  determined,  the  next  step  is  to 
proceed  to  the  supplementary  accuracy  analysis. 
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3. 3. 2. 2  Supplementary  Accuracy  Analysis 

If  no  rifle  selection  has  been  made  at  the  conclusion  of  the 
sustainability  analysis,  the  next  step  is  tc  repeat  the  primary  analy¬ 
sis  using  hit  probability  per  trigger  pill.  Since  no  differences 
occurred  in  terms  of  target  hits,  it  is  unlikely  that  significant 
differences  will  occur  here.  However,  before  proceeding  further,  it 
is  desirable  to  insure  that  weapon  performance  is  relatively  equal 
with  respect  to  range.  If  the  capabilities  varied  significantly  with 
respect  to  range,  it  is  likely  that  the  test  officer  would  be  aware 
of  a  potential  problem  before  this  stage  in  the  analysis  is  reached. 
This  analysis  will  provide  a  check  and  will  yield  hard  data  on  per¬ 
formance  with  respect  to  range  as  a  safeguard  against  a  possible 
discrepancy.  For  example,  such  a  problem  could  occur  if  a  shotgun 
or  other  short  range  weapon  were  compared  to  a  slow  rate  of  fire, 
highly  accurate  weapon.  The  latter  would  achieve  a  few  hits  con¬ 
sistently  over  the  entire  range  spectrum  producing  an  exponential 
curve.  A  rapid  firing  shotgun  could  produce  a  large  number  of  close 
range  hits  such  that  the  total  hits  would  be  equal  even  though  there 
is  a  decided  difference  in  capability.  This  supplementary  test  will 
provide  a  picture  of  performance  at  all  ranges.  This  output  will 
also  be  referred  to  during  the  optimization  analysis.  If  a  signifi¬ 
cant  difference  does  occur  during  this  test,  this  is  an  indicator 
that  the  test  scenario  was  unrealistic  in  terms  of  target  exposure. 
Close  range  targets  probably  were  allowed  to  remain  in  view  too  long, 
permitting  an  excessive  number  of  hits  at  close  range.  Before  pro¬ 
ceeding,  any  discrepancy  between  the  results  of  the  targets  hit 
analysis  and  the  hit  probability  analysis  will  have  to  be  explained. 
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3. 3. 2. 3  Responsiveness  Analysis 


The  responsiveness  analysis  is  associated  with  the  quickfire 
situation  only,  since  the  type  of  action  required  (quick  violent  re¬ 
sponse  to  target  cues)  is  inherent  in  the  attack  and  oefense  only 
to  a  limited  extent.  Therefore,  these  data  would  not  he  used  as  selec¬ 
tion  criteria  until  all  other  tests  failed  to  provide  a  basis  for 
weapon  selection.  Performance  in  the  attack  and  defensive  situations 
must  be  considered  before  responsiveness  data  ccn:e  into  consideration 
because  of  the  frequency  with  which  attack  and  defensive  combat  actions 
take  place  as  compared  to  quickfire  situations. 

The  responsiveness  analysis  consists  of  four  iterations  through 
a  two-way  analysis  using  responsiveness  HOE  (see  Figure  15).  If  one 
weapon  system  fails  to  show  superiority  with  all  four  MOE,  data  are 
combined  to  form  a  fifth  measure,  time  to  shift  fire  and  achieve  a 
second  hit.  This  combat  task  requires  the  test  soldier  to  hit  two 
targets  in  rapid  succession.  This  approach  is  based  on  the  analysis 
of  the  quickfire  1  data  which  indicated  that  the  two  best  discrimi¬ 
nators  were  time  to  first  hit  and  time  to  shift  fire.  This  new 
measure  combines  both  of  these  and  should  discriminate  if  there  is 
any  real  significance  between  weapons. 

If  after  proceeding  through  this  entire  secondary  analysis  no 
selection  has  been  made,  it  can  be  safely  assumed  that  no  real  opera¬ 
tional  performance  differences  exist.  The  analyst  should  proceed  to 
the  optimization  analysis  to  improve  the  standard  rifle's  performance 
to  the  extent  that  time  permits.  If  a  selection  has  been  made  prior 
to  reaching  this  point,  the  optimization  analysis  already  has  begun 
witli  tne  superior  weapon  system,  either  the  standard  of  the  test  rifle. 
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3-4-  Optimization  Analysis 

3.4.1  Selection  of  MOE 

All  available  MOE  are  candidates  for  use  in  the  optimization 
analysis.  The  analysis  is  designed  to  present  as  complete  a  picture 
of  rifle  performance  as  possible  to  insure  that  the  operational 
characteristics  are  thoroughly  understood.  The  analysis  should  indi¬ 
cate  strengths  and  weaknesses  of  the  weapon  system.  The  MOE  recom¬ 
mended  for  use  are  summarized  below: 


3. 4. 1.1  Attack  Facility  MOE 

a.  All  hit  probabilities 

b.  Miss  distance 

c.  Time  between  rounds 

d.  Time  between  hits 

e.  Time  to  first  hit 

f.  Movement  times 

3 .4. 1.2  Defense  Facility  MOE 

a.  All  hit  probabilities 

b.  Miss  distance 

c.  Time  to  shift  fire 

d.  Time  to  change  magazines 

e.  Time  to  recharge  magazines 

f .  Time  to  clear  malfunctions 

3. 4. 1.3  Quickfire  Facility 

a.  Time  to  first  round 

b.  Time  to  first  hit 

c.  Hit  probability  -  semi¬ 
automatic 

d.  Hit  probability  -automatic 


g.  Time  to  shift  fire 

h.  Time  to  change  magazine 

i.  Number  of  rounds  to  first  hit 

j.  Ease  of  handling  in  assault 
role 

k.  Compatibility  with  other  equip¬ 
ment 

g.  Number  of  rounds  between  mal¬ 
functions 

h.  Sound  level 

i.  Light  reduction 

j.  Visual  light  emission 

k.  jyection  pattern 

^  d.  Time  between  trigger  pulls 

f.  Time  to  shift  fire 

g.  Rounds  fired 

h.  Miss  distance 


3.4.2  Technical  Approach 


3- 4. 2.1  Introduction  -  The  technical  approach  assumes  that  in  any 
comparative  test  between  rifles,  the  rifle  which  is  not  selected  as 
the  superior  weapon  system  will  produce  data  which  are  valuable  in 
comparing  the  relative  strengths  and  weaknesses  of  the  two  weapon 
systems.  In  fact,  the  "inferior1'  rifle  may  exceed  the  selected  rifle 
in  performance  with  some  MOE.  The  comparing  of  performance,  then,  is 
a  useful  method  of  determining  operating  characteristics  which  could 
possibly  be  improved.  For  example,  in  Figure  6,  a  performance  profile 
is  shown  for  two  ’weapons.  The  X-axis  is  the  percent  difference  between 
the  weapons  and  the  Y-axis  contains  a  set  of  selected  MOE.  The  graph 
shows  that  the  selected  weapon  is  generally  superior  with  the  excep¬ 
tions  in  time  to  shift  fire  and  burst  hit  probability.  These  would 
be  indicators  that  possibly  some  characteristics  from  the  inferior 
rifle  could  be  transferred  to  the  selected  weapon  to  improve  its 
responsiveness  capability.  The  following  paragraphs  describe  several 
viseful  techniques  which  are  available  for  isolating  meaningful  differ¬ 
ences  in  close  competing  weapon  systems  using  quantitative  MOE.  The 
more  qualitative  MOE  (visual  light  emission,  ejection  patterns)  must 
be  analysed  subjectively  and  are  not  included  in  this  analysis. 

3. 4. 2. 2  Pictorial  Analysis  -  The  first  analytical  technique  for 
optimising  weapon  system  performance  is  to  prepsi’e  graphs  showing 
weapon  system  performance  as  a  function  of  weapon,  mode,  range,  and 
facility  using  a  variety  of  MOE.  The  analyst  must  search  for  anomalies 
in  the  data  base,  which  will  appear  as  extreme  changes  in  curve  shape 
or  extreme  fluctuation  from  point  to  point.  As  performance  is  examined 
as  a  function  of  range,  for  instance,  the  analyst  should  note  the 
characteristics  of  the  curve.  Is  it  smooth?  Does  it  reach  the  x-axis 
prematurely  indicating  poor  performance  at  extreme  ranges?  Does  per¬ 
formance  change  appreciably  from  one  test  facility  to  another?  For 
instance,  a  longer  time  between  rounds  on  the  quickfire  facility  might 
indicate  difficulty  in  handling  the  weapon  when  used  in  the  off-hand 
position.  Potential  problems  noted  by  the  test  officer  are  another 
indicator  of  where  to  look  in  the  data  base  for  performance  problems. 
Further,  side-by-side  plots  of  performance  curves  for  the  two  weapons 
systems  are  often  useful  in  isolating  particular  problems. 

3. 4. 2. 3  Effectiveness  Analysis  -  Another  useful  technique  is  to 
segregate  engagements  by  some  criterion  such  as  effectiveness.  For 
example,  define  effectiveness  ns  the  .ability  to  achieve  a  hit  on  a 
target  and  designate  these  engagements  as  successful.  Divide  all  engage¬ 
ments  on  a  particular  facility  into  two  categories:  successful  and 
unsuccessful.  Plot  various  MOE  on  graph  paper  to  determine  if  some 

MOE  behave  differently  under  the  two  conditions.  During  Quickfire 
Experiment  I,  for  instance,  burst  si^c  was  found  to  be  an  important 
factor.  Multivariate  analysis  is  an  expeditious  means  of  isolating 
these  differences.  The  same  result  can  be  obtained  by  employing 
several  sequential  sorting  stops  on  the  data  base.  For  instance,  after 
sorting  on  weapon  and  mode  to  get  all  automatic  engagem-  ’.vs  for  one 
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RIFLE  A 


RIFLE  B 


PERCENT  DIFFERENCE 


RIFLE  PERFORMANCE  PROFILE 
Figure  16 


weapon,  sort  on  hits  to  subdivide  the  set  into  successful  and  unsuccess¬ 
ful  categories  within  these  subtests,  calculate  means  for  burst  size, 
time  to  first  rounds,  etc.,  to  find  differences  in  behavior  as  a  func¬ 
tion  of  degree  of  success.  Once  a  problem  is  identified,  recommenda¬ 
tion  to  correct  the  problem  and  improve  performance  can  be  made.  Such 
recommendations  might  include  a  change  in  stock  design,  sighting  sys¬ 
tem,  ammunition  carrying  pouches,  sling  arrangement,  magazine  sice,  or 
burst  size.  In  many  cases,  problems  will  be  identified  bub  cause  and 
effect  will  not  be  established.  If  the  problem  is  important  enough, 
subsequent,  controlled  tests  may  be  required  to  determine  the  exact 
cause  of  the  problem.  The  maiiy  KOE  available  can  provide  a  very  com¬ 
plete  picture  of  weapon  system  performance  and  should  be  used  to  the 
extent  possible  to  produce  this  picture. 

3. 4. 2. 4  Quart ile  Analysis  -  Another  technique  is  called  a  quartile 
analysis.  The  effectiveness  analysis  section  above  was  concerned  with 
engagements.  The  quartile  analysis,  although  similar,  focuses  on  the 
effective  soldiers.  In  this  analysis  the  most  effective  soldiers 
(upper  25$)  and  least  effective  soldiers  (lower  25$)  are  pulled  from 
the  data  base.  The  effectiveness  criteria  are  the  number  of  hits 
achieved  on  a  specific  facility.  Once  the  two  groups  have  been  iso¬ 
lated  the  KOE  pertinent  to  that  facility  will  be  calculated  for  each 
set  of  individuals.  A  series  of  t-tests  are  run  on  each  MOE  to  compare 
the  two  quartiles.  In  addition  to  a  t-test  comparison  across  groups, 
a  correlation  analysis  among  KOE  both  within  groups  and  between  groups 
is  suggested.  The “correlation  analysis  can  include  pnysical  character¬ 
istics  in  addition  to  performance  data  to  determine  the  possible  effect 
of  left-handedness,  height,  visual  acuity  or  other  human  factors. 

Figure  17  shows  the  quickfire  MOE  and  the  letter  in  the  right  hand 
column  references  paragraphs  below  which  attempt  to  define  the  meaning 
of  significant  differences. 


Quickfire  Facility 
MOE 


Lower 

Quartile  (X) 


Upper 

Quartile  (X) 


Significance* 

(t-test) 


Time  to  first  round 

Time  to  first  hit 

Hit  probability 

Time  between  trigger  pulls 

Time  to  shift  fire 

Rounds  fired 

Miss  distance 


A 

B 

C 

D 

E 

F 

G 


Quartile  Analysis  for  Quickfire  Facility  MOE 

Figure  17 


♦Significance : 

A  -  Time  to  First  Round  -  A  significant  difference  in  favor  of  the 
lower  quartile  would  imply  that  the  lower  quartile  were  rushing 
their  shots  and  that  possibly  more  training  could  improve  their 
performance.  A  significant  difference  in  favor  of  the  upper 
quartile  would  imply  one  of  two  things  -  the  lower  group  was 
having  either  target  acquisition  difficulties  or  interface 
problems.  It  would  be  advisable  to  do  a  human  factors  corre¬ 
lations  analysis  to  determine  if  some  physical  attribute  could 
be  causing  the  problem  (size,  poor  vision,  etc.). 

B  -  Time  to  First  Hit  -  The  same  implications  described  above  should 
apply  to  this  MOE. 

C  -  Hit  Probability  -  The  upper  and  lower  extremes  provided  in  this 
analysis  is  a  measure  of  human  variation.  If  the  variation  is 
smaller  for  the  inferior  weapon  than  for  the  selected  weapon, 
this  indicates  a  possible  human  interface  problem  with  the 
sights,  recoil,  or  target  obscuration.  Possibly  more  training 
is  indicated. 
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D  -  Time  Between  Trigger  Pulls  -  A  significant  difference  would 
imply  interface  problems  in  coping  with  recoil  and  realigning 
the  sights.  The  problem  could  be  influenced  by  some  physi¬ 
cal  attribute  that  one  quartile  has  in  common.  A  human 
factors  analysis  is  indicated.  Possibly  improvements  could 
be  found  in  training  methods  or  weapon  design. 

E  -  Time  to  Shift  Fire  -  The  comments  under  D  above  generally 

apply  to  this  MOE;  emphasis  should  be  placed  on  vision  since, 
target  acquisition  is  an  added  variable. 

F  -  Rounds  Fired  -  A  correlation  with  hit  probability  that  was 
not  significant  would  indicate  that  the  lower  quartile  was 
simply  not  firing  a  sufficient  number  of  rounds  at  the  tar¬ 
get.  The  problem  may  be  due  to  acquisition  which  again 
emphasizes  the  importance  of  human  factors.  A  significant 
positive  correlation  would  indicate  that  the  lower  quartile 
consists  of  a  set  of  soldiers  who  are  simply  less  adept  at 
handling  fire  arms. 

G  -  Miss  Distance  -  This  measure  of  spread  is  related  to  human 
factors  and  interface  problems.  A  high  "negative”  correla¬ 
tion  with  hit  probability,  that  is,  a  low  mean  spread  and 
a  high  hit  probability,  should  be  expected.  No  correlation 
or  a  negative  correlation  would  indicate  potential  improve¬ 
ment  either  through  solving  the  interace  problem  or  improv¬ 
ing  training. 

Figure  l8  shows  the  MOE  for  the  attack  facility  and  the  letters  refer 
to  subparagraphs  below  which  suggest  possible  interpretations  of 
specific  meaningful  differences. 
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Quartile  Analysis  for  Attack  Facility  MOE 


Figure  18 


A  -  Hit  Probability  -  Reference  quickfire  facility,  paragraph 
C  above. 

B  -  Miss  Distance  -  Reference  quickfire  facility,  paragraph  G 
above. 

C  -  Time  Between  Trigger  Pulls  -  The  interpretation  of  signifi¬ 
cant  differences  on  the  attack  facility  is  slightly  differ¬ 
ent  than  on  the  quickfire  facility.  On  the  attack  facility 
a  fast  reaction,  quick  response  is  not  necessarily  required, 
and  firing  is  more  deliberate  with  emphasis  on  accuracy  at 
the  longer  ranges  and  ammunition  conservation.  The  test 
soldier  must  judge  the  time  to  complete  the  attack  with 
reference  to  his  basic  load.  Rate  of  expenditure  of  basic 
load  is  a  measure  of  the  adequacy  of  the  basic  load.  Rate 
of  expenditure  is  also  an  indicator  of  the  potential  combat 
time  of  the  weapon  system.  If  the  mean  for  the  upper  quar- 
tile  is  extremely  high  in  terms  of  ammunition  available,  a 
potential  sustainability  problem  is  indicated.  A  broad 
spread  between  groups  would  he  indicative  of  a  potential  train- 
ing  problem  since  all  test  soldiers  should  be  expected  to 
behave  similarly  in  an  identical  situation.  During  the 
assault  phase  of  the  attack,  this  MOE  takes  on  more  of  the 
responsiveness  cnaracteristics  described  in  quickfira  para¬ 
graph  D  above. 

D  -  Number  of  rounds  Between  Hits  -  The  spread  between  means  of 
the  two  quartiles  would  be  an  indicator  of  the  importance 
the  test  soldiers  place  on  getting  rounds  down  range  and  the 
accuracy  of  the  weapon  system.  A  high  correlation  with  hit 
probability  within  groups  would  indicate  the  latter;  a  low 
correlation  indicates  the  former.  Again,  an  extreme  spread 
would  indicate  training  problems. 

E  -  Movement  Times  -  This  measure  should  indicate  potential 
problems  in  physical  characteristics  among  test  soldiers. 

A  correlation  with  various  physical  attributes  should  be 
performed, 

F  -  Magazine  Change  Time  -  A  large  spread  between  quartiles  would 
indicate  training  problems.  Compared  across  rifles,  a  large 
spread  would  indicate  a  potential  problem  with  che  magazine 
component  of  the  weapon  system,  e.g.,  pouch,  pouch  closure, 
seating  and  removing  the  magazine. 
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Figure  19  shows  MOE  for  the  defense  facility  and  the  letters  refer  to 
subparagraphs  below; 
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Quartile  Analysis  for  Defense  Facility  MOE 
Figure  l9 

A  -  Hit  Probability  -  See  quickfire  facility,  Paragraph  C. 

B  -  Miss  Distance  -  See  quickfire  facility,  Paragraph  G. 

C  -  Time  to  First  Hit  -  See  quickfire  facility,  Paragraph  B.  The 
measure  has  slightly  less  impact  on  the  defense  facility  due 
to  the  more  deliberate  firing  method, 

D  -  Time  to  Shift  Fire  -  See  quickfire  facility,  Paragraph  E. 

This  measure  has  slightly  less  impact  on  the  defense  facility 
due  to  the  more  deliberate  firing  method. 

E  -  Time  Between  Trigger  Pulls  -  See  attack  facility,  Paragraph  C. 

F  -  Time  to  Change  Magazines  -  See  attack  facility,  Paragraph  F. 

G  -  Time  to  Clear  Malfunctions  -  A  significant  difference  between 
quartiles  may  be  indicative  of  a  training  problem  in  that 
some  test  soldiers  are  not  adept  in  the  actions  of  reducing 


a  malfunction,  especially  if  poor  performance  with  this 
measure  does  not  correlate  with  poor  over-all  performance. 

H  -  Number  of  Rounds  between  Malfunctions  -  Again,  a  signif¬ 
icant  difference  may  indicate  a  training  problem  such  as 
failing  to  seat  the  magazine  properly  or  failing  to  keep  the 
magazine  clean.  Further,  it  may  indicate  nonuniformity 
between  weapons;  that  is,  certain  weapons  are  more  prone 
to  malfunction  than  other  weapons  of  the  same  type  due  to 
quality  control  or  specific  parts. 

3. 4. 2. 5  Multivariate  Analysis  -  The  final  technique  for  optimizing 
weapon  performance  is  multivariate  discriminant  analysis.  This  tech¬ 
nique  was  useful  for  sifting  through  the  large  amounts  of  information 
generated  by  an  operational  service  test  to  search  for  meaningful 
measures.  The  independent  or  test  variables  are  weapon,  mode  of  fire, 
range,  and  facility.  The  dependent  variables  are  the  various  measures 
of  effectiveness.  The  recommended  method  is  to  begin  the  analysis 
with  all  applicable  MOE  for  each  set  of  test  variables  and  to  reduce 
the  number  of  MOE  in  subsequent  analysis  by  eliminating  MOE  that  fail 
to  correlate  with  the  discriminant  function.  The  procedure  terminates 
when  the  best  four  or  five  MOE  are  found  for  each  set  of  test  variables. 
The  results  are  then  interpreted  in  a  manner  similar  to  that  described 
in  the  preceding  paragraphs. 

3. 4. 2. 6  A  Precautionary  Note  -  The  MOE  are  indicators  of  weapon 
system  performance  and  will  pinpoint  strengths  and  weaknesses  of  a 
system.  The  MOE  will  not  necessarily  provide  the  causes  of  those 
strengths  and  weaknesses.  Cause  and  effect  relationships  at  this  stage 
are  somewhat  subjective.  If  the  cause  is  not  readily  identifiable, 
subsequent  testing  may  be  required  to  identify  the  cause  objectively. 
Factors  that  weigh  in  the  decision  to  continue  testing  are  costs,  time, 
and  the  commitment  of  other  resources  versus  the  potential  gain  from 
isolating  the  cause  and  in^lementing  corrective  action. 


